SELFormer: Molecular Representation Learning via SELFIES Language Models

Abstract Automated computational analysis of the vast chemical space is critical for numerous fields research such as drug discovery and material science. Representation learning techniques have recently been employed with primary objective generating compact informative numerical expressions complex data, efficient usage in subsequent prediction tasks. One approach to efficiently learn molecular representations processing string-based notations chemicals via natural language algorithms. Majority methods proposed so far utilize SMILES this purpose, which most extensively used encoding molecules. However, associated problems related validity robustness, may prevent model from effectively uncovering knowledge hidden data. In study, we propose SELFormer, a transformer architecture-based (CLM) that utilizes 100% valid, expressive notation, SELFIES, input, order flexible high-quality representations. SELFormer pre-trained on two million drug-like compounds fine-tuned diverse property Our performance evaluation has revealed that, outperforms all competing methods, including graph learning-based approaches SMILES-based CLMs, predicting aqueous solubility molecules adverse reactions, while producing comparable results remaining We also visualized learned by dimensionality reduction, indicated even can discriminate differing structural properties. shared programmatic tool, together its datasets models at https://github.com/HUBioDataLab/SELFormer . Overall, our demonstrates benefit using SELFIES context modeling opens up new possibilities design novel candidates desired features.